{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The [Containers](Containers.ipynb) tutorial shows examples of each of the container types in HoloViews, and it is useful to look at the description of each type there, as you work through this tutorial.  \n",
    "\n",
    "This tutorial shows you how to combine the various container types, in order to build data structures that can contain all of the data that you want to visualize or analyze, in an extremely flexible way.  For instance, you may have a large set of measurements of different types of data (numerical, image, textual notations, etc.) from different experiments done on different days, with various different parameter values associated with each one.  HoloViews can store all of this data together, which will allow you to select just the right bit of data \"on the fly\" for any particular analysis or visualization, by indexing, slicing, selecting, and sampling in this data structure.\n",
    "\n",
    "To illustrate the full functionality provided, we will create an example of the maximally nested object structure currently possible with HoloViews:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import holoviews as hv\n",
    "hv.notebook_extension()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.seed(10)\n",
    "\n",
    "def sine_curve(phase, freq, amp, power, samples=102):\n",
    "    xvals = [0.1* i for i in range(samples)]\n",
    "    return [(x, amp*np.sin(phase+freq*x)**power) for x in xvals]\n",
    "\n",
    "phases =      [0, np.pi/2, np.pi, 3*np.pi/2]\n",
    "powers =      [1,2,3]\n",
    "amplitudes =  [0.5,0.75, 1.0]\n",
    "frequencies = [0.5, 0.75, 1.0, 1.25, 1.5, 1.75]\n",
    "\n",
    "\n",
    "gridspace = hv.GridSpace(kdims=['Amplitude', 'Power'], group='Parameters', label='Sines')\n",
    "\n",
    "for power in powers:\n",
    "    for amplitude in amplitudes:\n",
    "        holomap = hv.HoloMap(kdims=['Frequency'])\n",
    "        for frequency in frequencies:\n",
    "            sines = {phase : hv.Curve(sine_curve(phase, frequency, amplitude, power))\n",
    "                     for phase in phases}\n",
    "            ndoverlay = hv.NdOverlay(sines , kdims=['Phase']).relabel(group='Phases',\n",
    "                                                                      label='Sines', depth=1)\n",
    "            overlay = ndoverlay * hv.Points([(i,0) for i in range(0,10)], group='Markers', label='Dots')\n",
    "            holomap[frequency] = overlay\n",
    "        gridspace[amplitude, power] = holomap\n",
    "\n",
    "penguins = hv.RGB.load_image('../assets/penguins.png').relabel(group=\"Family\", label=\"Penguin\")\n",
    "\n",
    "layout = gridspace + penguins"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This code produces what looks like a relatively simple animation of two side-by-side figures, but is actually a deeply nested data structure:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "layout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The structure of this object can be seen using ``print()``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(layout)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Nesting hierarchy <a id='NestingHierarchy'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To help us understand this structure, here is a schematic for us to refer to as we unpack this object, level by level:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<center><img src=\"https://assets.holoviews.org/nesting-diagram.png\"></center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Everything that is *displayable* in HoloViews has this same basic structure, although any of the levels can be omitted in simpler cases, and many different Element types (not containers) can be substituted for any other.  \n",
    "\n",
    "Since HoloViews 1.3.0, you are allowed to build data-structures that violate this hierarchy (e.g., you can put ``Layout`` objects into ``HoloMaps``) but the resulting object cannot be displayed. Instead, you will be prompted with a message to call the ``collate`` method. Using the ``collate`` method will allow you to generate the appropriate object that correctly obeys the hierarchy shown above, so that it can be displayed.\n",
    "\n",
    "As shown in the diagram, there are three different types of container involved:\n",
    "\n",
    "- Basic Element: elementary HoloViews object containing raw data in an external format like Numpy or pandas.\n",
    "- Homogeneous container (UniformNdMapping): collections of Elements or other HoloViews components that are all the same type.  These are indexed using array-style key access with values sorted along some dimension(s), e.g. ``[0.50]`` or ``[\"a\",7.6]``.\n",
    "- Heterogenous container (AttrTree): collections of data of different types, e.g. different types of Element.  These are accessed by categories using attributes, e.g. ``.Parameters.Sines``, which does not assume any ordering of a dimension.\n",
    "\n",
    "We will now go through each of the containers of these different types, at each level."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ``Layout`` Level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Above, we have already viewed the highest level of our data structure as a Layout. Here is the representation of entire Layout object, which reflects all the levels shown in the diagram:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(layout)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the examples below, we will unpack this data structure using attribute access (explained in the [Introductory tutorial](Introduction.ipynb)) as well as indexing and slicing (explained in the [Sampling Data tutorial](Sampling_Data.ipynb)).\n",
    "\n",
    "### ``GridSpace`` Level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Elements within a ``Layout``, such as the ``GridSpace`` in this example, are reached via attribute access:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "layout.Parameters.Sines"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ``HoloMap`` Level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This ``GridSpace`` consists of nine ``HoloMap``s arranged in a two-dimensional space.  Let's now select one of these ``HoloMap`` objects, by indexing to retrieve the one at [Amplitude,Power] ``[0.5,1.0]``, i.e. the lowest amplitude and power:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "layout.Parameters.Sines[0.5, 1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As shown in the schematic above, a ``HoloMap`` contains many elements with associated keys. In this example, these keys are indexed with a dimension ``Frequency``, which is why the ``Frequency`` varies when you play the animation here."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ``Overlay`` Level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The printed representation showed us that the ``HoloMap`` is composed of ``Overlay`` objects, six in this case (giving six frames to the animation above).  Let us access one of these elements, i.e. one frame of the animation above, by indexing to retrieve an ``Overlay`` associated with the key with a ``Frequency`` of *1.0*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "layout.Parameters.Sines[0.5, 1][1.0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NdOverlay Level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As the representation shows, the ``Overlay`` contains a ``Points`` object and an ``NdOverlay`` object.  We can access either one of these using the attribute access supported by ``Overlay``:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(layout.Parameters.Sines[0.5, 1][1].Phases.Sines +\n",
    " layout.Parameters.Sines[0.5, 1][1].Markers.Dots)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ``Curve`` Level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The ``NdOverlay`` is so named because it is an overlay of items indexed by dimensions, unlike the regular attribute-access overlay types.  In this case it is indexed by ``Phase``, with four values.  If we index to select one of these values, we will get an individual ``Curve``, e.g. the one with zero phase:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "l=layout.Parameters.Sines[0.5, 1][1].Phases.Sines[0.0]\n",
    "l"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(l)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data Level"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, we have reached the end of the HoloViews objects; below this object is only the raw data as a Numpy array:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "type(layout.Parameters.Sines[0.5, 1][1].Phases.Sines[0.0].data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Actually, HoloViews will let you go even further down, accessing data inside the Numpy array using the continuous (floating-point) coordinate systems declared in HoloViews. E.g. here we can ask for a single datapoint, such as the value at x=5.2:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "layout.Parameters.Sines[0.5, 1][1].Phases.Sines[0.0][5.2]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Indexing into 1D Elements like Curve and higher-dimensional but regularly gridded Elements like Image, Surface, and HeatMap will return the nearest defined value (i.e., the results \"snap\" to the nearest data item):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "layout.Parameters.Sines[0.5, 1][1].Phases.Sines[0.0][5.23], layout.Parameters.Sines[0.5, 1][1].Phases.Sines[0.0][5.27]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For other Element types, such as Points, snapping is not supported and thus indexing down into the .data array will be less useful, because it will only succeed for a perfect floating-point match on the key dimensions.  In those cases, you can still use all of the access methods provided by the numpy array itself, via ``.data``, e.g. ``.data[52]``, but note that such native operations force you to use the native indexing scheme of the array, i.e. integer access starting at zero, not the more convenient and semantically meaningful [continuous coordinate systems](Continuous_Coordinates.ipynb) we provide through HoloViews."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Indexing using ``.select``"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The curve displayed immediately above shows the final, deepest Element access possible in HoloViews for this object:\n",
    "\n",
    "```python\n",
    "layout.Parameters.Sines[0.5, 1][1].Phases.Sines[0.0]\n",
    "```\n",
    "This is the curve with an amplitude of *0.5*, raised to a power of *1.0* with frequency of *1.0* and *0* phase. These are all the numbers, in order, used in the access shown above.\n",
    "\n",
    "The ``.select`` method is a more explicit way to use key access, with both of these equivalent to each other:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "o1 = layout.Parameters.Sines.select(Amplitude=0.5, Power=1.0).select(Frequency=1.0)\n",
    "o2 = layout.Parameters.Sines.select(Amplitude=0.5, Power=1.0, Frequency=1.0)\n",
    "o1 + o2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The second form demonstrates HoloViews' **deep indexing** feature, which allows indexes to cross nested container boundaries. The above is as far as we can index before reaching a heterogeneous type (the ``Overlay``), where we need to use attribute access. Here is the more explicit method of indexing down to a curve, using ``.select`` to specify dimensions by name instead of bracket-based indexing by position:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "layout.Parameters.Sines.select(Amplitude=0.5,Power=1.0, \n",
    "                               Frequency=1.0).Phases.Sines.select(Phase=0.0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Summary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see, HoloViews lets you compose objects of heterogenous types, and objects covering many different numerical or other dimensions, laying them out spatially, temporally, or overlaid.  The resulting data structures are complex, but they are composed of simple elements with well-defined interactions, making it feasible to express nearly any relationship that will characterize your data.  In practice, you will probably not need this many levels, but given this complete example, you should be able to construct an appropriate organization for whatever type of data that you do want to organize or visualize. \n",
    "\n",
    "As emphasized above, it is not possible to combine these objects in other orderings.  Of course, any ``Element`` can be substituted for any other, which doesn't change the structure.  But you cannot e.g. display an ``Overlay`` or ``HoloMap`` of ``Layout`` objects.  Confusingly, the objects may *act* as if you have these arrangements.  For instance, a ``Layout`` of ``HoloMap`` objects will be animated, like ``HoloMap`` objects, but only because of the extra dimension(s) provided by the enclosed ``HoloMap`` objects, not because the ``Layout`` itself has data along those dimensions.  Similarly, you cannot have a ``Layout`` of ``Layout`` objects, even though it looks like you can.  E.g. the ``+`` operator on two ``Layout`` objects will not create a ``Layout`` of ``Layout`` objects; it just creates a new ``Layout`` object containing the data from both of the other objects.  Similarly for the ``Overlay`` of ``Overlay`` objects using ``*``; only a single combined ``Overlay`` is returned.\n",
    "\n",
    "If you are confused about how all of this works in practice, you can use the examples in the tutorials to guide you, especially the [Exploring Data](Exploring_Data.ipynb) tutorial.  "
   ]
  }
 ],
 "metadata": {
  "language_info": {
   "name": "python",
   "pygments_lexer": "ipython3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}
